The present invention relates to graphical programming, and in particular to a system and method for converting a graphical program into a hardware implementation. The present invention further relates to a system and method for configuring an instrument or device, such as an image acquisition or machine vision device, a motion control device, or a programmable logic controller, to perform measurement functions, wherein the instrument or device includes a programmable hardware element
Traditionally, high level text-based programming languages have been used by programmers in writing applications programs. Many different high level programming languages exist, including BASIC, C, FORTRAN, Pascal, COBOL, ADA, APL, etc. Programs written in these high level languages are translated to the machine language level by translators known as compilers or interpreters. The high level programming languages in this level, as well as the assembly language level, are referred to as text-based programming environments.
Increasingly computers are required to be used and programmed by those who are not highly trained in computer programming techniques. When traditional text-based programming environments are used, the user""s programming skills and ability to interact with the computer system often become a limiting factor in the achievement of optimal utilization of the computer system.
There are numerous subtle complexities which a user must master before he can efficiently program a computer system in a text-based environment. The task of programming a computer system to model or implement a process often is further complicated by the fact that a sequence of mathematical formulas, mathematical steps or other procedures customarily used to conceptually model a process often does not closely correspond to the traditional text-based programming techniques used to program a computer system to model such a process. In other words, the requirement that a user program in a text-based programming environment places a level of abstraction between the user""s conceptualization of the solution and the implementation of a method that accomplishes this solution in a computer program. Thus, a user often must substantially master different skills in order to both conceptually model a system and then to program a computer to model that system. Since a user often is not fully proficient in techniques for programming a computer system in a text-based environment to implement his model, the efficiency with which the computer system can be utilized to perform such modeling often is reduced.
Examples of fields in which computer systems are employed to interact with physical systems are the fields of instrumentation, process control, industrial automation, and simulation. Computer measurement and control of devices such as instruments or industrial automation hardware has become increasingly desirable in view of the increasing complexity and variety of instruments and devices available for use. However, due to the wide variety of possible testing and control situations and environments, and also the wide array of instruments or devices available, it is often necessary for a user to develop a custom program to control a desired system.
As discussed above, computer programs used to control such systems traditionally had to be written in text-based programming languages such as, for example, assembly language, C, FORTRAN, BASIC, etc. Traditional users of these systems, however, often were not highly trained in programming techniques and, in addition, text-based programming languages were not sufficiently intuitive to allow users to use these languages without training. Therefore, implementation of such systems frequently required the involvement of a programmer to write software for control and analysis of instrumentation or industrial automation data. Thus, development and maintenance of the software elements in these systems often proved to be difficult.
U.S. Pat. Nos. 4,901,221; 4,914,568; 5,291,587; 5,301,301; and 5,301,336; among others, to Kodosky et al disclose a graphical system and method for modeling a process, i.e., a graphical programming environment which enables a user to easily and intuitively model a process. The graphical programming environment disclosed in Kodosky et al can be considered a higher and more intuitive way in which to interact with a computer. A graphically based programming environment can be represented at a level above text-based high level programming languages such as C, Basic, Java, etc.
The method disclosed in Kodosky et al allows a user to construct a diagram using a block diagram editor. The block diagram may include a plurality of interconnected icons such that the diagram created graphically displays a procedure or method for accomplishing a certain result, such as manipulating one or more input variables and/or producing one or more output variables. In response to the user constructing a diagram or graphical program using the block diagram editor, data structures may be automatically constructed which characterize an execution procedure which corresponds to the displayed procedure. The graphical program may be compiled or interpreted by a computer.
Therefore, Kodosky et al teaches a graphical programming environment wherein a user places or manipulates icons and interconnects or xe2x80x9cwires upxe2x80x9d the icons in a block diagram using a block diagram editor to create a graphical xe2x80x9cprogram.xe2x80x9d A graphical program for performing an instrumentation, measurement or automation function, such as measuring a Unit Under Test (UUT) or device, controlling or modeling instruments, controlling or measuring a system or process, or for modeling or simulating devices, may be referred to as a virtual instrument (VI). Thus, a user can create a computer program solely by using a graphically based programming environment. This graphically based programming environment may be used for creating virtual instrumentation systems, modeling processes, control, simulation, and numerical analysis, as well as for any type of general programming.
A graphical program may have a graphical user interface. For example, in creating a graphical program, a user may create a front panel or user interface panel. The front panel may include various graphical user interface elements or front panel objects, such as user interface controls and/or indicators, that represent or display the respective input and output that will be used by the graphical program or VI, and may include other icons which represent devices being controlled. The front panel may be comprised in a single window of user interface elements, or may comprise a plurality of individual windows each having one or more user interface elements, wherein the individual windows may optionally be tiled together. When the controls and indicators are created in the front panel, corresponding icons or terminals may be automatically created in the block diagram by the block diagram editor. Alternatively, the user can place terminal icons in the block diagram which may cause the display of corresponding front panel objects in the front panel, either at edit time or later at run time. As another example, the front panel may comprise front panel objects, e.g., the GUI, embedded in the block diagram.
During creation of the block diagram portion of the graphical program, the user may select various function nodes or icons that accomplish his desired result and connect the function nodes together. For example, the function nodes may be connected in one or more of a data flow, control flow, and/or execution flow format. The function nodes may also be connected in a xe2x80x9csignal flowxe2x80x9d format, which is a subset of data flow. The function nodes may be connected between the terminals of the various user interface elements, e.g., between the respective controls and indicators. Thus the user may create or assemble a graphical program, referred to as a block diagram, graphically representing the desired process. The assembled graphical program may be represented in the memory of the computer system as data structures and/or program instructions. The assembled graphical program, i.e., these data structures, may then be compiled or interpreted to produce machine language that accomplishes the desired method or process as shown in the block diagram.
Input data to a graphical program may be received from any of various sources, such as from a device, unit under test, a process being measured or controlled, another computer program, or from a file. Also, a user may input data to a graphical program or virtual instrument using a graphical user interface, e.g., a front panel as described above. The input data may propagate through the data flow block diagram or graphical program and appear as changes on the output indicators. In an instrumentation application, the front panel can be analogized to the front panel of an instrument. In an industrial automation application the front panel can be analogized to the MMI (Man Machine Interface) of a device. The user may adjust the controls on the front panel to affect the input and view the output on the respective indicators. Alternatively, the front panel may be used merely to view the input and output, or just the output, and the input may not be interactively manipulable by the user during program execution.
Thus, graphical programming has become a powerful tool available to programmers. Graphical programming environments such as the National Instruments LabVIEW product have become very popular. Tools such as LabVIEW have greatly increased the productivity of programmers, and increasing numbers of programmers are using graphical programming environments to develop their software applications. In particular, graphical programming tools are being used for test and measurement, data acquisition, process control, man machine interface (MMI), supervisory control and data acquisition (SCADA) applications, simulation, image processing/machine vision applications, and motion control, among others.
A primary goal of graphical programming, including virtual instrumentation, is to provide the user the maximum amount of flexibility to create his/her own applications and/or define his/her own instrument functionality. In this regard, it is desirable to extend the level at which the user is able to program a device, e.g., extend the level at which a user of instrumentation or industrial automation hardware is able to program an instrument. The evolution of the levels at which the user has been able to program an instrument is essentially as follows.
1. User level software (LabVIEW, LabWindows CVI, Visual Basic, etc.)
2. Kernel level software
3. Auxiliary kernel level software (a second kernel running along side the main OS, e.g., InTime, VentureCom, etc.)
4. Embedded kernel level software
5. Hardware level software (FPGAxe2x80x94the present patent application)
In general, going down the above list, the user is able to create software applications which provide a more deterministic real-time response. Currently, most programming development tools for instrumentation or industrial automation provide an interface at level 1 above. In general, most users are unable and/or not allowed to program at the kernel level or auxiliary kernel level. The user level software typically takes the form of software tools that can be used to create software which operates at levels 1 and/or 4.
Current instrumentation solutions at level 5 primarily exist as vendor-defined solutions, i.e., vendor created modules. However, it would be highly desirable to provide the user with the ability to develop user level software which operates at the hardware level. More particularly, it would be desirable to provide the user with the ability to develop high level software, such as graphical programs, which can then be readily converted into hardware level functionality. This would provide the user with the dual benefits of being able to program device functionality at the highest level possible (text-based or graphical programs), while also providing the ability to have the created program operate directly in hardware for increased speed and efficiency.
One embodiment of the present invention comprises a computer-implemented system and method for automatically generating hardware level functionality, e.g., configured programmable hardware such as FPGAs or CPLDs, in response to a graphical program created by a user. This provides the user the ability to develop or define desired functionality using graphical programming techniques, while enabling the resulting program to operate directly in hardware. The computer-implemented system and method may be used for configuring an instrument to perform a measurement function, wherein the instrument includes the programmable hardware element.
The user may first create a graphical program which performs or represents the desired functionality. The graphical program may comprise a block diagram which includes a plurality of interconnected nodes which visually indicate functionality of the graphical program. The plurality of nodes may be interconnected in one or more of a data flow, control flow, or execution flow format. The graphical program may include a single diagram or a hierarchy of subprograms or sub-diagrams. In one embodiment, the user may place various constructs in portions of the graphical program to aid in conversion of these portions into hardware form. As the user creates or assembles the graphical program on the display, data structures and/or software code may be automatically created and stored in memory corresponding to the graphical program being created. In a measurement or instrumentation application, the graphical program may implement a measurement function.
The user may then select an option to convert the graphical program into executable form, wherein at least a portion of the graphical program is converted into a hardware implementation. In one embodiment, the block diagram portion of the graphical program is converted into a hardware configuration that may be downloaded onto the programmable hardware element comprised on the instrument or measurement device. If the graphical program includes a user interface portion, then the user interface portion may not be converted into a hardware configuration, but rather may be compiled for execution by a processor.
The user may optionally select which portions of a graphical program, e.g., which diagrams (or sub-VIs), are to be translated into hardware form, either during creation of the graphical program or when selecting the option to convert the graphical program into executable form. Thus the user can select a first portion of the graphical program, e.g., a first one or more diagrams which require real time execution, to be downloaded onto the programmable hardware element, and may select other diagrams which do not require real time execution to be executed by a processor. Alternatively, the selection of portions of the graphical program to be compiled for execution by the CPU or to be provided for hardware implementation may be automatically performed by the system.
The portion of the graphical program selected for hardware implementation may first be converted to a an abstract hardware graph, also referred to as a VDiagram tree. More specifically, the data structures corresponding to the graphical program may be converted into the abstract hardware graph or VDiagram tree. Thus in one embodiment the conversion may not be a one-step process of generating a hardware description directly from graphical program internal data structures, but rather may involve the VDiagram intermediate format. The VDiagram tree may comprise data structures representing the functional objects of the graphical program and the data flow, control flow or execution flow between them. VDiagrams are described in detail below.
The VDiagram tree may be parsed by a back end program module which generates a specific hardware description from the tree, such as a VHDL or EDIF hardware description. The hardware description may then converted into a hardware netlist. Netlists for various types of hardware devices may be created from a hardware description (e.g., FPGA-specific netlists, CPLD-specific netlists, etc.). As used herein, the term xe2x80x9cnetlistxe2x80x9d comprises various intermediate hardware-specific description formats comprising information regarding the particular hardware elements required to implement a hardware design and the relationship among those elements. For example, the back end may generate a VHDL file containing a hardware description of the graphical program. An FPGA-specific netlist may then be created from the VHDL file, wherein the netlist comprises information regarding the various FPGA components (and the logical relationship among those components) necessary to implement the hardware design described in the VHDL file.
The process of converting a hardware description such as a VHDL file into a netlist may be performed by readily available synthesis tools, as is well known in the art. The netlist may then be compiled into a hardware program file (also called a software bit stream) which may be used to program a programmable logic device (PLD) or programmable hardware element such as an FPGA or a CPLD, or other types of (re)configurable hardware devices. In one embodiment, the hardware description is converted into an FPGA program file.
The step of compiling the resulting netlist into a PLD program file may use a library of pre-compiled function blocks to aid in the compilation, as well as hardware target specific information. The library of pre-compiled function blocks may include netlist libraries for programmatic structures, such as for/next loops, while/do loops, case structures, and sequence structures, among others. This allows the user to program with high-level programming constructs, such as iteration, looping, and case structures, while allowing the resulting program to execute directly in hardware.
The resulting bit stream may then be transferred to a PLD or other (re)configurable hardware device such as an FPGA to produce a programmed hardware device equivalent to the graphical program or block diagram.
One embodiment of the invention comprises a general purpose computer system which includes a CPU and memory, and an interface card or device coupled to the computer system which includes a programmable hardware element or logic, such as an FPGA. The computer system may include a graphical programming system which is used to develop the graphical program. The computer system may also include software which is operable to convert the graphical program into an abstract hardware graph, and then convert the abstract hardware graph into a hardware description. The computer system further may include a synthesis tool which is used to compile the hardware description into a netlist, as well as other tools for converting the netlist into a PLD program file for uploading into the PLD. The computer system may further include a library of pre-compiled function blocks which are used by the synthesis tool to aid in creating the netlist.
As described above, in one embodiment, a user creates a graphical program and then the system and method of the present invention may operate to convert at least a portion of the graphical program into a hardware description. The hardware description may then be converted to a hardware program which executes on a PLD. The present system and method thus extends the traditional model of software development to include the possibility of running at least a portion (or all) of a graphical program on hardware, instead of running the entire graphical program on the general-purpose processor.
However, it is noted that the use of the present invention is not limited to the creation of application programs as described above. One embodiment comprises a system and method to create an abstract hardware graph from a graphical program. Various back end programs may be called to generate disparate types of hardware descriptions from the abstract hardware graph. These hardware descriptions may be used for purposes other than creating a bit stream for programming a PLD. For example, a hardware description may be used in the process of creating and printing a traditional circuit board which may be produced in mass quantities.
It is also noted that various back end programs may be called to generate software source code, such as C language code, from the abstract hardware graph. As described in detail below, the abstract hardware graph (VDiagram tree) generated by the present method may contain information regarding the execution order and data flow of the graphical program. This information may be used to establish a procedural order in a traditional programming language. Also, since the execution order for portions of a graphical program may be inherently parallel, the system and method described herein are well-suited for creating a program in a text-based parallel programming language.
In one embodiment, during or after the user creates the graphical program, the method may then estimate and optionally display one or more of the size and cost of a hardware implementation of the graphical program. In one embodiment, for example where the graphical program implements a measurement function, the graphical program manipulates one or more hardware resources of the instrument. Examples of hardware resources include A/D converters, D/A converters, timers, counters, clocks, memories, pixel processors, etc. In this embodiment, creating the graphical program includes displaying an indication of usage, or the status of usage, of the one or more hardware resources during creation of the graphical program.
In another embodiment, the user may insert a probe at a location in the graphical program, wherein the probe is operable to display data generated at the location during execution of the graphical program. In this embodiment, the programmable hardware element includes the probe element for implementing probing in the programmable hardware element. It is noted that non real-time clock cycle observations are possible in current FPGA""s, such as Xilinx FPGAs, without external/explicit hooks. However, explicit probes/probe interfaces on the program of the FPGA allow external visibility and real-time performance.
In one embodiment, the target device including the programmable hardware element or reconfigurable hardware (or PLD) being programmed comprises an instrument or measurement device. The instrument or measurement device may be a xe2x80x9cstand-alonexe2x80x9d instrument, or a measurement device or card coupled to or comprised in a computer system. Example instruments or measurement devices include a data acquisition device or card, a GPIB interface card or GPIB instrument, a VXI interface card or VXI instrument card, a PXI card or device, an image processing (or machine vision) device or card, a smart camera, a smart sensor, a computer-based instrument device or card such as an oscilloscope or multimeter device, a motion control device or card, a traditional instrument such as a stand-alone oscilloscope or multimeter, or other measurement or sensing device. The target device including the reconfigurable hardware or PLD being programmed may also comprise an automation or control device, such as a programmable logic controller, a fieldbus card or device, a CAN (Controller Area Network) device or card, a distributed data acquisition device or module, or other type of automation or control device. The target device being programmed may comprise an instrument or device connected to the computer, such as through a serial connection. It is noted that the target instrument or device being programmed, which includes a programmable hardware element or other (re)configurable hardware element, can take any of various forms, as desired.
In the above embodiments, the user may create a graphical program or block diagram which implements a measurement function. For example, where the target device is a computer-based reconfigurable instrument, the measurement function may incorporate oscilloscope functionality to perform a desired measurement. Where the target device is am image acquisition device or smart camera, the measurement function may comprise an image processing or machine vision function that processes an image. Where the target device is a motion control device, the function may comprise a motion control algorithm specified by the user to control a motion device to move an object.
The method may then generate a hardware description based on at least a portion of the graphical program, wherein the hardware description describes a hardware implementation of the at least a portion of the graphical program. The programmable hardware element is then configured in the instrument or device utilizing the hardware description, wherein after configuration the programmable hardware element implements a hardware implementation of the at least a portion of the graphical program. After the hardware element has been configured, the system may be executed. During execution, the instrument may acquire a signal from an external source, and the programmable hardware element in the instrument may execute to perform the measurement function on the signal.
As one example, where the instrument is an image acquisition device or smart camera, the programmable hardware element in the image acquisition device or smart camera can be configured to perform an image processing (or machine vision) function. In this example embodiment, a graphical program may first be created, either by the user or automatically from another representation, such as a prototype. The graphical program may implement an image processing or machine vision function. The graphical program may include a user interface portion and a block diagram portion. A hardware description may then be generated based on the block diagram portion of the graphical program, wherein the hardware description describes a hardware implementation of the block diagram portion of the graphical program. The programmable hardware element in the image acquisition device or smart camera may then be configured utilizing the hardware description. After configuration, the programmable hardware element implements a hardware implementation of the block diagram portion of the graphical program.
During execution, a camera associated with the image acquisition device, or comprised in the smart camera, may acquire an image of an object. For example, the image acquisition device or smart camera may be used in a machine vision inspection application, or a manufacturing assembly application. The programmable hardware element in the image acquisition device or smart camera may then receive the acquired image of the object. The programmable hardware element in the image acquisition device or smart camera may then execute to perform the image processing function (or machine vision function) on the image. The results of this image processing may then be used to make a decision in a machine vision inspection application or may be used to perform an operation in a manufacturing assembly application.
During execution of the programmable hardware element in the image acquisition device or smart camera, the user interface portion may be presented on a display. A user or operator may use the displayed user interface portion to view or control the image processing function (or machine vision function).
Thus the method may operate to configure an instrument, measurement device, or control device to perform measurement functions, wherein the instrument includes a programmable hardware element.
As another example, where the instrument or device is a motion control device, the programmable hardware element in the motion control device can be configured to perform a motion function. In this example embodiment, a graphical program may first be created, either by the user or automatically from another representation, such as a prototype. The graphical program may implement a motion function, i.e., the graphical program may describe a motion sequence. The graphical program may include a user interface portion and a block diagram portion. A hardware description may then be generated based on the block diagram portion of the graphical program, wherein the hardware description describes a hardware implementation of the block diagram portion of the graphical program. The programmable hardware element in the motion control device may then be configured utilizing the hardware description. After configuration, the programmable hardware element implements a hardware implementation of the block diagram portion of the graphical program.
The motion control device may be used in a machine vision inspection application to move an object being inspected or to move a camera. The motion control device may also be used in a manufacturing assembly application to move a part or object being assembled, or may be used in a robotics application, among others. During the respective application, the programmable hardware element in the motion control device may then execute to move the motion control device accordingly.
During execution of the programmable hardware element in the motion control device, the user interface portion may be presented on a display. A user or operator may use the displayed user interface portion to view or control the motion control function.
Thus the method may operate to configure an instrument (which includes various types of measurement devices and/or control devices) to perform measurement functions, wherein the instrument includes a programmable hardware element.